Multi-Touch Integrated Desktop Environment

ABSTRACT

A technique for integrating a multi-touch surface into a desktop environment is disclosed. One or more multi-touch display devices are placed on the horizontal surface surrounding or beneath a keyboard or mouse. At least one region on the multi-touch surface is defined to display one or more user interface tools on the multi-touch surface. The one or more user interface tools displayed via the multi-touch surface may control or display information associated with desktop applications displayed via a primary display device.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates generally to input/output devices and, morespecifically, to a multi-touch integrated desktop environment.

2. Description of the Related Art

Multi-touch displays are becoming increasingly common in consumerdevices. For example, over the past decade, desktop monitors andhand-held devices have incorporated multi-touch surfaces with whichusers can interact to provide input information to the underlyingdevices. One interesting aspect of a multi-touch surface is the abilityto control multiple degrees-of-freedom. For example, conventionalrectangle manipulations include translation, rotation, and scalingoperations performed on a multi-touch surface by dragging, rotating, orexpanding/contracting, respectively, one or more contact points. Thus, auser may interact with an application via a multi-touch surface bytapping, dragging, expanding, contracting, or rotating one or morecontact points on the multi-touch surface. These operations aretypically more efficient to perform on a multi-touch surface than withtraditional input/output devices.

Typically, when a multi-touch surface is used within a desktopenvironment, the multi-touch surface replaces the input/output devicesone would typically use with a desktop machine. For example, atraditional keyboard is replaced with an on-screen keyboard projected onthe multi-touch surface, a traditional mouse is replaced with amulti-touch surface on the primary display or by a touchpad, and astylus is added for inputting text on the multi-touch surface viasoftware that recognizes handwriting.

One drawback to this conventional design approach is that multi-touchsurfaces suffer from sensing resolution problems that can limit theprecision of touch input. Consequently, actions such as typing on aprojected keyboard or selecting fine features in a drawing applicationmay be difficult using a multi-touch surface. Another drawback is thatan on-screen keyboard lacks tactile feedback to the user and, therefore,users may not be as comfortable typing using the on-screen keyboardwithout the feeling of depressing each individual keystroke.

As the foregoing illustrates, what is needed in the art is a system thatmore optimally integrates the benefits of a multi-touch surface into atraditional desktop environment.

SUMMARY OF THE INVENTION

One embodiment of the invention sets forth a method for integrating amulti-touch surface into a desktop environment. The method generallyincludes detecting a location of at least one input device on themulti-touch surface and defining at least one region on the multi-touchsurface in proximity to the location of the at least one input device.The method further includes monitoring the multi-touch surface for anychanges in the location of the at least one input device and, inresponse to a change in the location of the at least one input device,changing the location of the at least one region to substantially matchthe change in the location of the at least one input device.

Another embodiment of the invention sets forth a method for configuringa multi-touch surface to display an enhanced task bar. The methodgenerally includes defining a region on the multi-touch surfaceproximate to one or more input devices detected on the multi-touchsurface for displaying the enhanced task bar and generating one or moreuser interface elements that are disposed within the enhanced task bar,where each user interface element is configured to control a differentapplication window displayed via a display device. The method furtherincludes displaying the enhanced task bar in the region on themulti-touch surface, detecting multi-touch input associated with a firstuser interface element disposed within the enhanced task bar, andadjusting a first application window displayed via the display devicebased on the multi-touch input.

Yet another embodiment of the invention sets forth a method forconfiguring a multi-touch surface to display a multi-functional touchpad. The method generally includes defining a region on the multi-touchsurface proximate to one or more input devices detected on themulti-touch surface for displaying the multi-functional touch pad andgenerating one or more user interface elements that are disposed withinthe multi-functional touch pad, where each user interface element isconfigured to control a different operation within an application windowdisplayed via a display device. The method further includes displayingthe multi-functional touch pad in the region on the multi-touch surface,detecting multi-touch input associated with a first user interfaceelement disposed within the multi-functional touch pad, and performing afirst operation within the application window based on the multi-touchinput.

Yet another embodiment of the invention sets forth a method forconfiguring a multi-touch surface to display a digital mouse pad. Themethod generally includes defining a region on the multi-touch surfaceproximate to a mouse device detected on the multi-touch surface fordisplaying the digital mouse pad and generating one or more userinterface elements that are disposed within the digital mouse pad, whereeach user interface element is configured to control a differentoperation within an application window displayed via a display device.The method further includes displaying the digital mouse pad in theregion on the multi-touch surface, detecting multi-touch inputassociated with a first user interface element disposed within thedigital mouse pad, and performing a first operation within theapplication window based on the multi-touch input.

One advantage of the disclosed techniques is that multi-touch surfacesare integrated with the desktop environment without removing thetraditional keyboard and mouse used for precision input. A user maystill type using a keyboard or use a mouse for precision selection. Inaddition, the user may utilize multi-touch regions in proximity to thekeyboard and mouse to perform operations better suited to multi-touchinput. In this manner, the bandwidth of interaction between the user andthe computer interface is increased.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the inventioncan be understood in detail, a more particular description of theinvention, briefly summarized above, may be had by reference toembodiments, some of which are illustrated in the appended drawings. Itis to be noted, however, that the appended drawings illustrate onlytypical embodiments of this invention and are therefore not to beconsidered limiting of its scope, for the invention may admit to otherequally effective embodiments.

FIG. 1 is a block diagram of a system configured to implement one ormore aspects of the present invention;

FIGS. 2A and 2B are a top-view and front-view, respectively, of amulti-touch desktop environment, according to one embodiment of thepresent invention;

FIG. 2C is a top-view of a multi-touch desktop environment, according toyet another embodiment of the present invention;

FIG. 3A illustrates a schematic diagram of multi-touch desktopenvironment of FIGS. 2A and 2B, according to one embodiment of thepresent invention;

FIG. 3B illustrates a schematic diagram of the multi-touch desktopenvironment of FIG. 3A with the keyboard and mouse removed, according toanother embodiment of the present invention;

FIGS. 3C through 3E illustrate schematic diagrams of the multi-touchdesktop environment of FIG. 3A, according to various alternativeembodiments of the present invention;

FIG. 4 is a flow diagram of method steps for configuring a multi-touchsurface in the multi-touch desktop environment, according to oneembodiment of the present invention;

FIG. 5 illustrates an enhanced task bar, according to one embodiment ofthe present invention;

FIG. 6 is a flow diagram of method steps for configuring an enhancedtask bar in a multi-touch desktop environment, according to oneembodiment of the present invention;

FIGS. 7A-7C illustrate aspects of a multi-functional touch pad,according to various embodiments of the present invention;

FIG. 8 is a flow diagram of method steps for configuring amultifunctional touch pad in a multi-touch desktop environment,according to one embodiment of the present invention;

FIG. 9 illustrates a digital mouse pad, according to one embodiment ofthe present invention; and

FIG. 10 is a flow diagram of method steps for configuring a digitalmouse pad in a multi-touch desktop environment, according to oneembodiment of the present invention.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth toprovide a more thorough understanding of the invention. However, it willbe apparent to one of skill in the art that the invention may bepracticed without one or more of these specific details. In otherinstances, well-known features have not been described in order to avoidobscuring the invention.

System Overview

FIG. 1 is a block diagram of a system 100 configured to implement one ormore aspects of the present invention. System 100 may be a computerworkstation, personal computer, video game console, or any other devicesuitable for practicing one or more embodiments of the presentinvention.

As shown, system 100 includes one or more processing units, such ascentral processing unit (CPU) 102, and a system memory 104 communicatingvia a bus path that may include a memory bridge 105. CPU 102 includesone or more processing cores, and, in operation, CPU 102 is the masterprocessor of system 100, controlling and coordinating operations ofother system components. System memory 104 stores software applicationsand data for use by CPU 102. CPU 102 runs software applications andoptionally an operating system. Memory bridge 105, which may be, e.g., aNorthbridge chip, is connected via a bus or other communication path(e.g., a HyperTransport link) to an I/O (input/output) bridge 107. I/Obridge 107, which may be, e.g., a Southbridge chip, receives user inputfrom one or more user input devices such as keyboard 108 or mouse 109and forwards the input to CPU 102 via memory bridge 105. In alternativeembodiments, I/O bridge 107 may also be connected to other input devicessuch as a joystick, digitizer tablets, touch pads, touch screens, stillor video cameras, motion sensors, and/or microphones (not shown).

One or more display processors, such as display processor 112, arecoupled to memory bridge 105 via a bus or other communication path 113(e.g., a PCI Express, Accelerated Graphics Port, or HyperTransportlink); in one embodiment display processor 112 is a graphics subsystemthat includes at least one graphics processing unit (GPU) and graphicsmemory. Graphics memory includes a display memory (e.g., a frame buffer)used for storing pixel data for each pixel of an output image. Graphicsmemory can be integrated in the same device as the GPU, connected as aseparate device with the GPU, and/or implemented within system memory104.

Display processor 112 periodically delivers pixels to a primary displaydevice 110. In one embodiment, primary display device 110 may be anyconventional CRT or LED monitor. Display processor 112 can provideprimary display device 110 with an analog or digital signal. Inalternative embodiments, primary display device 110 may comprise amulti-touch display device such as any conventional CRT or LED monitorwith an integrated sensor that detects the presence and location of auser touching the display area of the monitor. In such alternativeembodiments, primary display device 110 may provide gesture recognitioninput to display processor 112 or CPU 102.

Display processor 112 also periodically delivers pixels to and receivesgesture recognition input from a secondary multi-touch display device111. Secondary multi-touch display device 111 may be a desktop surfaceimplemented with a multi-touch surface, such as Microsoft Surface™. Inoperation, the secondary multi-touch display device 111 defines thetouch-sensitive portions of the desktop surface and enables displayprocessor 112 to display graphical content and provide gesturerecognition capabilities to system 100. In one embodiment, secondarymulti-touch display device 111 corresponds to the entire desktop surfaceavailable to the end-user. In other embodiments, secondary multi-touchdisplay device 111 may constitute only a portion of the desktop surface.

A system disk 114 is also connected to I/O bridge 107 and may beconfigured to store content and applications and data for use by CPU 102and display processor 112. System disk 114 provides non-volatile storagefor applications and data and may include fixed or removable hard diskdrives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray, HD-DVD, orother magnetic, optical, or solid state storage devices.

A switch 116 provides connections between I/O bridge 107 and othercomponents such as a network adapter 118 and various add-in cards 120and 121. Network adapter 118 allows system 100 to communicate with othersystems via an electronic communications network, and may include wiredor wireless communication over local area networks and wide areanetworks such as the Internet.

Other components (not shown), including USB or other port connections,film recording devices, and the like, may also be connected to I/Obridge 107. For example, an audio processor may be used to generateanalog or digital audio output from instructions and/or data provided byCPU 102, system memory 104, or system disk 114. Communication pathsinterconnecting the various components in FIG. 1 may be implementedusing any suitable protocols, such as PCI (Peripheral ComponentInterconnect), PCI Express (PCI-E), AGP (Accelerated Graphics Port),HyperTransport, or any other bus or point-to-point communicationprotocol(s), and connections between different devices may use differentprotocols, as is known in the art.

In one embodiment, display processor 112 incorporates circuitryoptimized for graphics and video processing, including, for example,video output circuitry, and constitutes a graphics processing unit(GPU). In another embodiment, display processor 112 incorporatescircuitry optimized for general purpose processing. In yet anotherembodiment, display processor 112 may be integrated with one or moreother system elements, such as the memory bridge 105, CPU 102, and I/Obridge 107 to form a system on chip (SoC). In still further embodiments,display processor 112 is omitted and software executed by CPU 102performs the functions of display processor 112.

Pixel data can be provided to display processor 112 directly from CPU102. In some embodiments of the present invention, instructions and/ordata representing a scene are provided to a render farm or a set ofserver computers, each similar to system 100, via network adapter 118 orsystem disk 114. The render farm generates one or more rendered imagesof the scene using the provided instructions and/or data. These renderedimages may be stored on computer-readable media in a digital format andoptionally returned to system 100 for display. Similarly, stereo imagepairs processed by display processor 112 may be output to other systemsfor display, stored in system disk 114, or stored on computer-readablemedia in a digital format.

Alternatively, CPU 102 provides display processor 112 with data and/orinstructions defining the desired output images, from which displayprocessor 112 generates the pixel data of one or more output images,including characterizing and/or adjusting the offset between stereoimage pairs. The data and/or instructions defining the desired outputimages can be stored in system memory 104 or a graphics memory withindisplay processor 112. In an embodiment, display processor 112 includes3D rendering capabilities for generating pixel data for output imagesfrom instructions and data defining the geometry, lighting shading,texturing, motion, and/or camera parameters for a scene. Displayprocessor 112 can further include one or more programmable executionunits capable of executing shader programs, tone mapping programs, andthe like.

In one embodiment, application 150 is stored in system memory 104.Application 150 may be any application configured to display a graphicaluser interface on both primary display device 110 and secondarymulti-touch display device 111. Application 150 may be configured tomove graphics objects between the primary display device 110 and thesecondary multi-touch display device 111.

It will be appreciated that the system shown herein is illustrative andthat variations and modifications are possible. The connection topology,including the number and arrangement of bridges, may be modified asdesired. For instance, in some embodiments, system memory 104 may beconnected to CPU 102 directly rather than through a bridge, and otherdevices may communicate with system memory 104 via memory bridge 105 andCPU 102. In other alternative topologies display processor 112 may beconnected to I/O bridge 107 or directly to CPU 102, rather than tomemory bridge 105. In still other embodiments, I/O bridge 107 and memorybridge 105 may be integrated in a single chip. In addition, theparticular components shown herein are optional. For instance, anynumber of add-in cards or peripheral devices might be supported. In someembodiments, switch 116 is eliminated, and network adapter 118 andadd-in cards 120, 121 connect directly to I/O bridge 107.

FIGS. 2A and 2B are a top-view and front-view, respectively, of amulti-touch desktop environment 200, according to one embodiment of thepresent invention. As shown, the multi-touch desktop environment 200includes, without limitation, a primary display device 110, a secondarymulti-touch display device 111, a keyboard 108, and a mouse 109. In oneembodiment, the primary display device 110 is positioned vertically on adesktop surface facing the end-user. The secondary multi-touch displaydevice 111 is positioned horizontally on a desktop surface between theprimary display device 110 and the end-user. The keyboard 108 and mouse109 may be positioned on top of the secondary multi-touch display device110.

In one embodiment, the front surface of primary display device 110implements a touch sensitive surface that includes multi-touch gesturerecognition capabilities. An end-user may use one or more fingers toperform various gestures that initiate operations such as rotate, scale,and zoom. In addition, the desktop surface includes secondarymulti-touch display 111 that also implements a touch-sensitive surfacethat includes multi-touch gesture recognition capabilities. Both theprimary display device 110 and the secondary multi-touch display device111 include the capability to display graphics and video on themulti-touch surface.

In alternative embodiments, primary display device 110 may be aconventional display device without multi-touch input surfaces. In suchembodiments, multi-touch desktop environment 200 only recognizesmulti-touch input on secondary multi-touch display device 111, and anyinput associated with primary display device 110 is performed usingconventional input devices such as keyboard 108 or mouse 109.

In one embodiment, multi-touch desktop environment 200 is configured totrack the position of keyboard 108 and mouse 109 on the secondarymulti-touch display device 111. While tracking the position of thekeyboard 108 and mouse 109, multi-touch desktop environment 200 maydisplay various tools on secondary multi-touch display device 111 thatenhance end-user interactivity such as by providing a secondary cursoror enabling a user to select various operations in application 150without using conventional input devices such as the keyboard 108 ormouse 109. In one embodiment, multi-touch desktop environment 200 isconfigured to display tools in regions of the multi-touch surface to theleft, right, bottom, and top of keyboard 108. In alternativeembodiments, multi-touch desktop environment 200 may be configured toutilize less than the entire desktop surface such as by onlyimplementing a multi-touch surface to one side of keyboard 108. Thesecondary multi-touch display device 111 may have a static configuration(i.e., the locations of various tools displayed on secondary multi-touchdisplay device 111 are pre-defined and fixed) or a dynamic configurationthat changes based on the tracked positions of the one or more inputdevices detected on secondary multi-touch display device 111. Asdescribed in further detail in conjunction with FIGS. 3A-3E below,multi-touch desktop environment 200 may be configured to changeconfigurations based on the position of the keyboard 108 and mouse 109.

FIG. 2C is a top-view of a multi-touch desktop environment 210,according to yet another embodiment of the present invention. As shown,the multi-touch desktop environment 210 implements secondary multi-touchdisplay device 111 as a tablet device, such as an Apple iPad®, AppleiPhone®, multi-touch tablet computer, etc. In such embodiments, thesecondary multi-touch display device 111 is positioned horizontally on adesktop surface in proximity to the other input devices included indesktop environment 210. As shown, secondary multi-touch display device111 is positioned to the left of keyboard 108 in desktop environment 210and may be connected to multi-touch desktop environment 210 in anytechnologically feasible way, such as via a communications pathimplemented using any suitable protocols such as USB, Bluetooth, or WiFicommunications protocols.

In one embodiment, CPU 102 may be configured to display various tools onthe tablet device and receive multi-touch input from the tablet device.In other embodiments, the tablet device may include an applicationconfigured to display a user interface tool that receives multi-touchinput from the user and transmits the multi-touch input to CPU 102. Itwill be appreciated that one or more tablet devices may be included inmulti-touch desktop environment 210 and positioned in differentlocations around keyboard 108 and mouse 109. Thus, multiple tools may bedisplayed on the one or more tablet devices at different positionsrelative to the input devices 108 and 109.

FIG. 3A illustrates a schematic diagram of multi-touch desktopenvironment 200 of FIGS. 2A and 2B, according to one embodiment of thepresent invention. As shown, the secondary multi-touch display device111 is divided into one or more multi-touch regions that implementvarious touch-sensitive tools to enhance user interactivity with themulti-touch desktop environment 200. These touch-sensitive toolsinclude, without limitation, an enhanced task bar displayed in a firstregion 310 of the secondary multi-touch display device 111, amulti-functional touch pad displayed in a second multi-touch region 320,a digital mouse pad displayed in a third multi-touch region 330, and acontinuous workspace displayed in a fourth multi-touch region 340. Inone embodiment, an operating system executing on CPU 102 may define thesize and location of the multi-touch regions 310, 320, 330 and 340 ofsecondary multi-touch display device 111. In alternative embodiments,the size and location of the regions may be defined by one or moreapplications such as application 150.

In one embodiment, as described in greater detail below in conjunctionwith FIGS. 5 and 6, the enhanced task bar 500 implemented in the firstmulti-touch region 310 is configured to enhance user interaction withthe desktop graphics and application graphics displayed on primarydisplay device 110. As described in greater detail below in conjunctionwith FIGS. 7A-7C and 8, the multi-functional touch pad 700 implementedin the second multi-touch region 320 provides users with configurabletools within easy finger reach of the keyboard 108 to facilitateinteractions with the computer system coupled to the multi-touch desktopenvironment 200. For example, the multi-functional touch pad 700 mayinclude tools for operating one or more frequently used softwareapplications. As described in greater detail below in conjunction withFIGS. 9 and 10, the digital mouse pad 900 implemented in the thirdmulti-touch region 330 is an area on which a physical mouse 109 can beplaced. The digital mouse pad 900 is configured to track the movementsof the physical mouse 109 and to provide other functionality to theuser, such as a clipboard function.

As also shown, the fourth multi-touch region 340 may constitute acontinuous workspace used as an extension of the primary display device110. The end-user may drag windows from the primary display device 110to the secondary multi-touch display device 111 using a cursor or afinger. In one embodiment, when a user drags a window from the primarydisplay device 110 to the secondary multi-touch display device 111, anabstract version of the window is displayed in multi-touch region 340.In alternative embodiments, the full version of the window is displayedin multi-touch region 340.

In one embodiment, multi-touch region 340 may constitute all unused areaof the display surface of secondary multi-touch display device 111. Inalternative embodiments, multi-touch region 340 may be configured with aspecific size and location in the secondary multi-touch display device111.

One should note that the locations of the various touch-sensitive toolswithin the secondary multi-touch display device 111 may be changedaccording to user preferences. For example, a left-handed user may wishto place the digital mouse pad 900 on the left side of the secondarymulti-touch display device 111 and the multi-functional touch pad 700 onthe right side. In one embodiment, a particular layout configuration maybe stored for each user of the multi-touch desktop environment 200. Insuch an embodiment, the secondary multi-touch display device 111automatically adjusts to reflect a user's preferred layout configurationwhen the user logs in. A default layout configuration may be set forusers who have not designated their own preferred layout configurations.

FIG. 3B illustrates a schematic diagram of the multi-touch desktopenvironment 200 of FIG. 3A with the keyboard 108 and mouse 109 removed,according to another embodiment of the present invention. As shown, whenthe keyboard 108 and mouse 109 are removed from the multi-touch desktopenvironment 200 of FIG. 3A, multi-touch regions 310, 320, 330, and 340are hidden, and multi-touch region 350 automatically expands toessentially the entire display area of secondary multi-touch displaydevice 111. In alternative embodiments, multi-touch region 350 may beconfigured to have a particular size and location such that multi-touchregion 350 constitutes only a portion of the display area of secondarymulti-touch display device 111.

Multi-touch region 350 may be configured to display application graphicsin place of primary multi-touch display device 110. In one embodiment,the active window in primary multi-touch display device 110 isautomatically displayed in multi-touch region 350. For example, if a mapis displayed in the active window in primary multi-touch display device110, the map may be displayed in the entire display area of secondarymulti-touch display device 111 quickly and automatically simply bymoving keyboard 108 and mouse 109 off the secondary multi-touch displaydevice 111.

In addition to enabling two-handed interactions and augmenting keyboard108 and mouse 109 inputs, multi-touch region 350 may constitute acontinuous workspace. An end-user can freely move windows betweenprimary display device 110 and secondary multi-touch display device 111using fingers or a cursor to take advantage of extra display surface. Inone embodiment, when a window is dragged from primary display device 110to secondary multi-touch display device 111, the size of user-interface(UI) elements in the dragged window are enlarged to suit finger scaleinteraction. In addition, in one embodiment, UI elements may berearranged to be close to keyboard 108 to provide a better userinteraction with secondary multi-touch display device 111. In yetanother embodiment, windows being displayed in secondary multi-touchdisplay device 111 may shift from full versions to abstract versions toallow users to absorb the most useful information with a simple glance.

FIGS. 3C through 3E illustrate schematic diagrams of the multi-touchdesktop environment 200 of FIG. 3A, according to various alternativeembodiments of the present invention. The multi-touch regions 310, 320,330, and 340 may be configured to have a size and position relative tothe location of keyboard 108 and/or mouse 109. The multi-touch regions310, 320, 330, and 340 may also be configured such that the total areaof all regions is less than the total display area of secondarymulti-touch display device 111. For example, as shown in FIG. 3C,multi-touch regions 310 and 340 are sized to be essentially as long askeyboard 108 and located directly below and above keyboard 108,respectively. Similarly, multi-touch regions 320 and 330 are sized to beessentially as wide as keyboard 108 and located directly to the left andright of keyboard 108, respectively.

As shown in FIG. 3D, when keyboard 108 is repositioned on secondarymulti-touch display device 111, the location of multi-touch regions 310,320, 330, and 340 is adjusted accordingly. In one embodiment, whenkeyboard 108 is moved to the top of secondary multi-touch display device111 such that the area between the keyboard 108 and the top of thedisplay area of secondary multi-touch display device 111 is less thanthe area of multi-touch region 340, the location of multi-touch region340 may be adjusted to be displayed directly below keyboard 108. In suchembodiments, multi-touch regions 310, 320 and 330 may be positionedbelow, to the left, and to the right of region 340 such that multi-touchregion 340 essentially switches positions with keyboard 108 in therelative layout of the keyboard 108 and the various multi-touch regions.Also, in such embodiments, when the user moves keyboard 108 back downsuch that the area between the keyboard 108 and the top of the displayarea of secondary multi-touch display device 111 is more than the areaof multi-touch region 340, multi-touch regions 310, 320, 330, and 340are adjusted such that the original configuration of the multi-touchregions with respect to the input devices is restored.

The various multi-touch regions of multi-touch desktop environment 200may be individually associated with the location and orientation of thekeyboard 108 and mouse 109. For example, as shown in FIG. 3E,multi-touch regions 310, 320, and 340 are associated with keyboard 108and multi-touch region 330 is associated with mouse 109. As also shownin FIG. 3E, the orientation of the multi-touch regions 310, 320, 330,and 340 on secondary multi-touch display device 111 may be adjustedbased on the orientation of keyboard 108 or mouse 109, individually. Forexample, as keyboard 108 is rotated on secondary multi-touch displaydevice 111 with respect to an axis defined by the lower edge ofsecondary multi-touch display device 111, multi-touch regions 310, 320,and 340 are rotated by a corresponding amount such that the orientationof the multi-touch regions remains fixed relative to the orientation ofthe keyboard 108.

In one or more embodiments, secondary multi-touch display device 111does not cover substantially the entire desktop surface area. Forexample, one of the various tools may be implemented on a tablet devicesuch as an Apple iPad®, Apple iPhone®, multi-touch tablet computer, etc.In this example, the tablet device may be placed on the user's desk withthe touch screen monitor facing up and at a position that correspondswith one of the multi-touch regions 310, 320, 330 or 340 that isconveniently within reach of a user's fingers while the user operateskeyboard 108. The tablet device may implement one of the various tools,such as a multi-functional touchpad 700.

In other embodiments, two or more multi-touch display devices may beimplemented in multi-touch desktop environment 200 in place of secondarymulti-touch display device 111. For example, each of the two or moremulti-touch display devices may implement a separate tool and be locatedin a different location relative to keyboard 108. A first tablet deviceconfigured to implement a multi-functional touchpad 700 may be placed tothe left of the keyboard 108. Simultaneously, a second tablet deviceconfigured to implement an enhanced task bar 500 may be placed belowkeyboard 108. Additional tablet devices may be placed in the multi-touchdesktop environment 200 to implement other multi-touch tools. It will beappreciated that in such instances where tablet devices placed inproximity to the keyboard 108 are configured to implement parts of thepresent invention, the functionality of adjusting the multi-touchregions 310, 320, 330 and 340 in response to moving the keyboard 108 onthe secondary multi-touch display device 111 may not be enabled.

FIG. 4 is a flow diagram of method steps for configuring a multi-touchsurface in the multi-touch desktop environment 200, according to oneembodiment of the present invention. Although the method steps aredescribed in conjunction with the systems of FIGS. 1, 2A-2C, and 3A-3E,persons skilled in the art will understand that any system configured toperform the method steps, in any order, is within the scope of thepresent invention.

The method 400 begins at step 410, where a CPU 102 detects the presenceof one or more input devices 108 and 109 on the display surface of thesecondary multi-touch display device 111. At step 420, CPU 102configures the display area of secondary multi-touch display 111 intoone or more defined areas such as multi-touch regions 310, 320, 330 and340. In one embodiment, the size and location of the one or moremulti-touch regions 310, 320, 330 and 340 are dependent on the locationof the input devices 108 and 109 on the display surface of the secondarymulti-touch display device 111.

At step 430, CPU 102 is configured to display one or more tools in themulti-touch regions 310, 320, 330 and 340 of secondary multi-touchdisplay device 111. Various tools may include an enhanced task bar 500,a multi-functional touch pad 700, or a digital mouse pad 900. Otherregions may be implemented as a continuous workspace of primarymulti-touch display device 110. It will be appreciated that any types oftools may be implemented in the various display regions of secondarymulti-touch display device 111 and are within the scope of the presentinvention.

At step 440, CPU 102 monitors the location of the input devices 108 and109 on the display surface of the secondary multi-touch display device111. In one embodiment, CPU 102 may be configured to adjust the locationof the one or more multi-touch regions 310, 320, 330 and 340 of thesecondary multi-touch display device 111 in response to detecting thatthe location of the input devices 108 and 109 on the secondarymulti-touch display device 111 has changed. At step 450, CPU 102determines if the location of the input devices 108 and 109 has changed.If the location of the input devices 108 and 109 has not changed, thenCPU 102 waits until the user moves the input devices 108 and 109.However, if CPU 102 detects that the location of the input devices 108and 109 has changed, the method 400 proceeds to step 460.

At step 460, CPU 102 determines whether the input devices 108 and 109are detected on the secondary multi-touch display device 111. If theinput devices 108 and 109 are detected on the secondary multi-touchdisplay device 111, then method 400 proceeds to step 470, where CPU 102may be configured to adjust the location of the multi-touch regions 310,320, 330 and 340 in secondary multi-touch display device 111 in relationto the position of input devices 108 and 109.

Returning now to step 460, if CPU 102 does not detect the input devices108 and 109 on the secondary multi-touch display device 111, then method400 proceeds to step 480, where CPU 102 may be configured to reconfigurethe regions 310, 320, 330, and 340 in the secondary multi-touch displaydevice 111. In one embodiment, CPU 102 is configured to generate asingle region in the secondary multi-touch display device 111 thatcovers essentially the entire area of the display surface. Inalternative embodiments, CPU 102 may be configured to resize orreposition one or more of multi-touch regions 310, 320, 330, and 340 inthe secondary multi-touch display device 111. After step 480, method 400terminates.

It will be appreciated that system 100 may be configured to perform thesteps of method 400 whenever CPU 102 detects input devices placed on thedisplay surface of the secondary multi-touch display device 111. In thismanner, the CPU 102 may be configured to adjust secondary multi-touchdisplay device 111 between one or more configurations based on thepresence or location of input devices on the multi-touch surface.

The present invention contemplates a variety of tools that may beconfigured to be displayed in the regions of secondary multi-touchdisplay device 111. Three of those tools (enhanced task bar 500,multi-functional touch pad 700, and digital mouse pad 900) are describedin greater detail below. In alternative embodiments, other tools may beimplemented in various multi-touch regions of secondary multi-touchdisplay device 111.

Enhanced Task Bar

FIG. 5 illustrates an enhanced task bar 500, according to one embodimentof the present invention. In one embodiment, the enhanced task bar 500may be displayed in multi-touch region 310 of multi-touch desktopenvironment 200. As shown in FIG. 5, enhanced task bar 500 includes astart menu icon 510 and task bar 520 as well as a window managementregion 530. Start menu icon 510 provides users with access to aconventional start menu for initiating applications or opening documentssuch as the start menu included in Microsoft's Windows OperatingSystem™. Task bar 520 displays icons 521, 522, and 523 representingwindows displayed in primary display device 110. In one embodiment,icons 521, 522, and 523 represent windows minimized in primary displaydevice 110. In other alternative embodiments, icons 521, 522, and 523may represent shortcuts that are pinned to task bar 520.

Window management region 530 displays thumbnails 531, 532, and 533representing windows displayed in primary display device 110. In oneembodiment, the location and size of thumbnails 531, 532, and 533 inwindow management region 530 conveys the spatial location and size ofwindows in primary display device 110. Thumbnails 531, 532, and 533 maybe displayed partially transparent in window management region 530 sothat even if windows are partially or totally occluded by other windowsthe user is able to see where a window is located in primary displaydevice 110. In alternative embodiments, window management region 530 hasa wider aspect ratio than primary display device 110 thus allowingthumbnails 531, 532, and 533 to be spread out more than thecorresponding windows on primary display device 110. Therefore, usersmay see and access windows more freely in window management region 530compared to primary display device 110.

The enhanced task bar 500 enables users to simultaneously managemultiple windows with two hands using various multi-touch gestures. Forexample, a user may resize a window associated with thumbnail 531 bymoving two fingers together or apart on thumbnail 531 in windowmanagement region 530 to shrink or expand the size of the window. Bymoving two fingers together/apart horizontally, the window is resized inwidth only. By moving two fingers together/apart vertically, the windowis resized in height only. By moving two fingers together/apartdiagonally, the window is resized in both width and heightsimultaneously. Minimizing a window may be accomplished by flicking thethumbnail 531 down in window management region 530 such that thethumbnail 531 is hidden in window management region 530 and an icon 521representing the window is displayed on task bar 520. Maximizing thewindow may be accomplished by double tapping the thumbnail 531 in windowmanagement region 530 or flicking the icon 521 (if the window isminimized in primary multi-touch display device 110) up from the taskbar 520. A window in primary display device 110 may be moved by touchingand dragging the corresponding thumbnail 531 in window management region530. Many other multi-touch gestures may be configured to performoperations in enhanced task bar 500 and are within the scope of thepresent invention.

FIG. 6 is a flow diagram of method steps 600 for configuring an enhancedtask bar 500 in a multi-touch desktop environment 200, according to oneembodiment of the present invention. Although the method steps aredescribed in conjunction with the systems of FIGS. 1, 2A-2C, 3A-3E and5, persons skilled in the art will understand that any system configuredto perform the method steps, in any order, is within the scope of theinventions.

Method 600 begins at step 610, where CPU 102 generates an enhanced taskbar 500 user interface that includes a start menu icon 510, a task bar520, and a windows management region 530. At step 620, CPU 102 generatesone or more user interface elements associated with windows open in theprimary display device 110. In one embodiment, CPU 102 generates icons521, 522, and 523 representing windows minimized in primary displaydevice 110, and thumbnails 531, 532, and 533 representing windowsdisplayed in primary display device 110. At step 630, CPU 102 populatesthe enhanced task bar 500 with the user interface elements generated instep 620. Icons 521, 522, and 523 may be added to task bar 520, andthumbnails 531, 532, and 533 may be displayed in the windows managementregion 530. In one embodiment, the location and size of thumbnails 531,532, and 533 in window management region 530 conveys the spatiallocation and size of windows in primary display device 110. Inalternative embodiments, window management region 530 has a wider aspectratio than primary display device 110 to allow easier access tothumbnails for a user.

At step 640, CPU 102 causes enhanced task bar 500 to be displayed in amulti-touch region of the secondary multi-touch display device 111. Inone embodiment, the multi-touch region may be defined as an area locatedbelow keyboard 108 in secondary multi-touch display device 111, such asregion 310. Displaying the enhanced task bar 500 in proximity tokeyboard 108 enables a user to efficiently transition between keyboardinput and multi-touch input.

At step 650, CPU 102 detects multi-touch gestures in the enhanced taskbar 500. For example, if a user touches secondary multi-touch displaydevice 111 in the area corresponding to the start menu icon 510, thestart menu may be opened in primary display device 110. At step 660, CPUperforms an operation based on the multi-touch gesture detected inenhanced task bar 500. Multi-touch gestures associated with windowmanagement operations include moving two fingers together or apart overa thumbnail 531 to resize a window, double tapping the thumbnail 531 tomaximize the window, flicking a thumbnail 531 down to minimize thewindow, flicking an icon 521 up to restore a window, and dragging athumbnail 531 to move a window in primary display device 110. Othermulti-touch gestures associated with window management operations arecontemplated and within the scope of the present invention.

Multi-Functional Touch Pad

FIGS. 7A-7C illustrate aspects of a multi-functional touch pad 700,according to various embodiments of the present invention.Multi-functional touch pad 700 is configured to display various toolsdesigned to enhance user interactivity with applications executing onsystem 100. For example, as shown in FIG. 7A, multi-functional touch pad700 may be configured to display a tool for adjusting the speed or gainof a mouse 109. Toolbar 710 defines a region of multi-functional touchpad 700 that enables a user to adjust the speed of a mouse by movingtogether or pulling apart two fingers touching the multi-touch surface.Touch point 711 and touch point 712 represent two points touched by theuser within the region defined by toolbar 710. If the user moves touchpoint 711 towards touch point 712 then the mouse gain is decreased andthe mouse moves slower. In contrast, if the user moves touch point 711away from touch point 712 then the mouse gain is increased and the mousemoves faster. Toolbar 710 represents a tool that implements a singledegree of freedom.

In contrast, as also shown in FIG. 7A, multi-functional touch pad 700may implement a tool with two or more degrees of freedom. For example,multi-functional touch pad 700 may be configured to display a tool forperforming rotation and scale manipulations of objects displayed onprimary display device 110. Specifically, toolbar 720 implements a toolwith two degrees of freedom for manipulating an object. A first degreeof freedom enables a user to adjust the scale of the object by movingtouch point 721 towards or away from touch point 722. A second degree offreedom enables a user to also rotate the object simultaneously byrotating touch point 722 around touch point 721.

In different embodiments, multi-functional touch pad 700 may beconfigured to implement other tools including, but not limited to, audiovisual controls, a secondary cursor, or drawing tools and the like. Forexample, as shown in FIG. 7B, multi-functional touch pad 700 may beconfigured to display a customizable tool palette 730 that includesapplication-specific icons (731, 732, etc.) for selecting various toolswithin an application 150. In one embodiment, application 150 may be adrawing application that enables a user to draw on a digital canvas. Insuch a case, a user may select the tool associated with icon 731, suchas a rectangle tool, using multi-functional touch pad 700. In thismanner, the user may select the tool associated with one of the icons(731, 732, etc.) using the multi-functional touch pad 700 while keepingthe primary cursor in focus on the digital canvas. Customizable toolpalette 730 may serve as a repository for storing commonly used UIelements.

In alternative embodiments, customizable tool palette 730 may beconfigured to add or remove tools according to user preferences. Thus,if a user rarely uses a particular tool, then the user can remove thattool from customizable tool palette 730. Similarly, the user may add anew tool to customizable tool palette 730 to provide easy access to afrequently used tool in application 150. For example, a tool may beremoved from the customizable tool palette 730 by dragging the icon 731out of the multi-functional touch pad 700. In contrast, to add a tool tocustomizable tool palette 730, a user may flick a tool icon in the mainapplication window down in primary display device 110. In response tosuch a multi-touch gesture, the icon 731 automatically appears incustomizable tool palette 730. In alternative embodiments, a user maydrag the tool icon from primary display device 110 to secondarymulti-touch display device 111. The icon 731 then appears in acontinuous workspace in secondary multi-touch display device 111 such asthe continuous workspace defined in multi-touch region 340. From thecontinuous workspace, the user may then place the icon in thecustomizable tool palette 730 by dragging the icon 731 from thecontinuous workspace to the multi-functional touch pad 700.

In yet other embodiments, multi-functional touch pad 700 may beconfigured to display a tool for adjusting the font characteristics in atext editor (or other associated application). For example, as shown inFIG. 7C, toolbar 740 may be displayed in multi-functional touch pad 700.Toolbar 740 may be configured with digital buttons or other UI elementsthat enable a user to change a font's size, color, or style. Toolbar 740includes a horizontal slider 741 to adjust a font's size, digitalbuttons 742 to adjust a font's style, and digital buttons 743 to adjusta font's color. As shown, digital buttons 742 and 743 are configured tochange the font's style or color in response a user touching the digitalbuttons 742 or 743 on the multi-touch surface. It will be appreciatedthat digital buttons 742 or 743 may be other UI elements such as adrop-down list or a color palette.

Multi-functional touch pad 700 may be configured to display toolsassociated with the active window in primary display device 110. Thus,when a user switches the active window in primary display device 110,multi-functional touch pad 700 may be configured to switch theparticular tool displayed in the multi-touch region. For example, whenthe active window in primary display device 110 is a text editor,multi-functional touch pad 700 may be configured to display toolbar 740.However, if the user then switches the active window to a drawingapplication, multi-functional touch pad 700 may be configured to hidetoolbar 740 and display customizable tool palette 730. It will beappreciated that multi-functional touch pad 700 may be configured withany number of tools that provide a user with various functions for aplurality of applications. The list described above in conjunction withFIGS. 7A-7C is not to be construed as limiting, and tools other thanthose described are within the scope of the present invention.

FIG. 8 is a flow diagram of method steps 800 for configuring amultifunctional touch pad 700 in a multi-touch desktop environment 200,according to one embodiment of the present invention. Although themethod steps are described in conjunction with the systems of FIGS. 1,2A-2C, 3A-3E and 7A-7C, persons skilled in the art will understand thatany system configured to perform the method steps, in any order, iswithin the scope of the inventions.

The method 800 begins at step 810, where CPU 102 generates one or moreuser interfaces such as toolbar 710, toolbar 720, customizable toolpalette 730, or toolbar 740. Each user interface may be associated withone or more of the open applications in primary display device 110. Atstep 820, CPU 102 detects the active window in primary display device110. At step 830, CPU 102 determines whether one of the user interfacesgenerated in step 810 is associated with the application correspondingto the active window in primary display device 110. For example, in thecase where the user interface is customizable tool palette 730, CPU 102determines whether the application corresponding to the active window isa drawing application. If no user interface is associated with theapplication corresponding to the active window, method 800 returns tostep 820, where CPU 102 waits until focus is shifted to a differentwindow in primary display device 110.

Returning now to step 830, if one of the user interfaces is associatedwith the application corresponding to the active window, method 800proceeds to step 840, where the user interface is displayed in amulti-touch region of the secondary multi-touch display device 111. Inone embodiment, the multi-functional touchpad 700 is displayed inmulti-touch region 320 in secondary multi-touch display device 111. Inalternative embodiments, multi-functional touch pad 700 may be displayedin a different region of the secondary multi-touch display device 111such as multi-touch region 330.

At step 850, CPU 102 detects multi-touch gestures in themulti-functional touch pad 700. For example, in the case wheremulti-functional touch pad is customizable tool palette 730, a user maytouch secondary multi-touch display device 111 in the area correspondingto icon 731 to select the tool associated with icon 731 such as arectangle tool. The particular user interface generated formulti-functional touch pad 700 may define various multi-touch gesturesfor providing input. For example, customizable tool palette 730 may onlyrecognize point input. In contrast, toolbar 720 may recognize two degreeof freedom multi-touch input for performing rotation and scalingoperations. At step 860, CPU 102 performs an operation based on themulti-touch gesture detected in multi-functional touch pad 700.

Digital Mouse Pad

FIG. 9 illustrates a digital mouse pad 900, according to one embodimentof the present invention. The digital mouse pad 900 may be associatedwith a multi-touch region of secondary multi-touch display device 111.Digital mouse pad 900 tracks the location of mouse 109 on secondarymulti-touch display device 111. In one embodiment, digital mouse pad 900is configured to display user interface (UI) elements 910 in proximityto mouse 109. The UI elements may be configured to enable a user toperform various functions conventionally selected through theright-click menu. In alternative embodiments, UI elements 910 may remainfixed at a specific location on the secondary multi-touch display device111 such as around the perimeter of the multi-touch region associatedwith digital mouse pad 900.

As shown in FIG. 9, digital mouse pad 900 may display digital buttons911, 912, and 913 for copy, cut, and paste clipboard operations,respectively. For example, a user could highlight an object using themouse 109 and then touch the copy digital button 911 to copy thehighlighted object to the clipboard. In one embodiment, when a user addsan object to the clipboard, the object is displayed in a thumbnail (914,915, etc.) to the right of the mouse 109. Thumbnails may display bothtextual and graphical contents. The clipboard may store one or moreobjects associated with thumbnails (914, 915, etc.) displayed inproximity to the mouse 109 on secondary multi-touch display device 111.The user may use the paste digital button 913 to paste the last objectcopied to the clipboard at the current cursor location. The user mayalso touch the thumbnail (914, 915, etc.) associated with a particularobject on the clipboard to paste that object at the current cursorlocation.

It will be appreciated that digital mouse pad 900 enables a user toefficiently select operations associated with a mouse 109 that may havetaken multiple steps using conventional techniques. In addition, digitalmouse pad 900 enables a user to quickly view multiple objects stored inthe clipboard simultaneously and select a particular object to paste inan application from among a plurality of objects. It will also beappreciated that digital mouse pad 900 may be configured with any numberof UI elements that provide a user with various functions for aplurality of applications. The list described above in conjunction withFIG. 9 is not to be construed as limiting, and UI elements other thanthose described are within the scope of the present invention.

FIG. 10 is a flow diagram of method steps 1000 for configuring a digitalmouse pad 900 in a multi-touch desktop environment 200, according to oneembodiment of the present invention. Although the method steps aredescribed in conjunction with the systems of FIGS. 1, 2A-2C, 3A-3E and9, persons skilled in the art will understand that any system configuredto perform the method steps, in any order, is within the scope of theinventions.

The method 1000 begins at step 1010, where CPU 102 associates amulti-touch region of secondary multi-touch display device 111 with thedigital mouse pad 900. In one embodiment, multi-touch region 330 isassociated with digital mouse pad 900 such that the digital mouse pad900 is located to the right of keyboard 108. When mouse 109 moves out ofthe multi-touch region associated with the digital mouse pad 900,digital mouse pad 900 may be hidden. In alternative embodiments, CPU 102may be configured to track mouse 109 on secondary multi-touch displaydevice 111 to adjust the location of the multi-touch region associatedwith the digital mouse pad 900 such that the position of digital mousepad 900 remains fixed relative to mouse 109.

At step 1020, CPU 102 detects the location of mouse 109 on secondarymulti-touch display device 111. At step 1030, CPU 102 determines whethermouse 109 is located in the multi-touch region associated with digitalmouse pad 900. If mouse 109 is not located within the multi-touch regionassociated with digital mouse pad 109, then method 1000 terminates.However, if mouse 109 is located in the multi-touch region then method1000 proceeds to step 1040.

At step 1040, CPU 102 displays UI elements 910 on digital mouse pad 900.In one embodiment, UI elements 910 may include digital buttons 911, 912,and 913 for performing clipboard functions as well as thumbnails 914,915, etc. for displaying objects stored in a clipboard. UI elements 910may be displayed in proximity to the location of mouse 109 on secondarymulti-touch display device 111. In alternative embodiments, UI elements910 may be displayed around the perimeter of the multi-touch regionassociated with digital mouse pad 900.

At step 1050, CPU 102 detects a multi-touch gesture on digital mouse pad900. In one embodiment, CPU 102 is configured to detect point input at aposition on secondary multi-touch display device 111 that corresponds toone of the UI elements in digital mouse pad 900. For example, CPU 102may be configured to detect when a user touches digital button 911. Atstep 1060, CPU 102 performs an operation based on the multi-touchgesture detected in step 1040. Returning to the example involvingdigital button 911, digital mouse pad 900 may be configured to perform acopy operation for any object currently highlighted in primary displaydevice 110. After step 1060, method 1000 returns to step 1020, where CPU102 detects a new location of mouse 109 on secondary multi-touch displaydevice 111.

In sum, the techniques disclosed above integrate multi-touch displaysurfaces with a conventional desktop environment. A multi-touch desktopenvironment includes a primary display device and a secondarymulti-touch display device located horizontally on the desktop surfacebetween the primary display device and an end-user. A keyboard and mouseare located in proximity to the secondary multi-touch display device.The multi-touch desktop environment defines regions of the secondarymulti-touch display device in proximity to the keyboard and mouse fordisplaying various tools for managing applications running on primarydisplay device. These tools include an enhanced task bar, amulti-functional touch pad, and a digital mouse pad. In addition,secondary multi-touch display device may implement a continuousworkspace that effectively increases the display area of the primarymulti-touch display device.

One advantage of the disclosed technique is that multi-touch surfacesare integrated with the desktop environment without removing thetraditional keyboard and mouse used for precision input. A user maystill type using a keyboard or use a mouse for precision selection. Inaddition, the user may utilize multi-touch regions in proximity to thekeyboard and mouse to perform operations better suited to multi-touchinput. In this manner, the bandwidth of interaction between the user andthe computer interface is increased.

While the foregoing is directed to embodiments of the present invention,other and further embodiments of the present invention may be devisedwithout departing from the basic scope thereof. For example, aspects ofthe present invention may be implemented in hardware or software or in acombination of hardware and software. One embodiment of the presentinvention may be implemented as a program product for use with acomputer system. The program(s) of the program product define functionsof the embodiments (including the methods described herein) and can becontained on a variety of computer-readable storage media. Illustrativecomputer-readable storage media include, but are not limited to: (i)non-writable storage media (e.g., read-only memory devices within acomputer such as CD-ROM disks readable by a CD-ROM drive, flash memory,ROM chips or any type of solid-state non-volatile semiconductor memory)on which information is permanently stored; and (ii) writable storagemedia (e.g., floppy disks within a diskette drive or hard-disk drive orany type of solid-state random-access semiconductor memory) on whichalterable information is stored. Such computer-readable storage media,when carrying computer-readable instructions that direct the functionsof the present invention, are embodiments of the present invention.

In view of the foregoing, the scope of the present invention isdetermined by the claims that follow.

1. A computer-implemented method for configuring a multi-touch surfaceto display a multi-functional touch pad, the method comprising: defininga region on the multi-touch surface for displaying the multi-functionaltouch pad, wherein the region is proximate to one or more input devicesdetected on the multi-touch surface; generating one or more userinterface elements that are disposed within the multi-functional touchpad, wherein each user interface element is configured to control adifferent operation within an application window displayed via a displaydevice; displaying the multi-functional touch pad in the region on themulti-touch surface; detecting multi-touch input associated with a firstuser interface element disposed within the multi-functional touch pad;and performing a first operation within the application window based onthe multi-touch input.
 2. The method of claim 1, wherein the first userinterface element comprises a tool for manipulating an object within theapplication window.
 3. The method of claim 2, wherein, based on themulti-touch input, the tool is configured to control two or more degreesof freedom of the object based on the multi-touch input.
 4. The methodof claim 1, wherein the first user interface element comprises an icon,and the first operation is performed when the multi-touch input isreceived at a location on the multi-touch surface corresponding to thelocation of the first user interface element.
 5. The method of claim 1,further comprising: detecting second multi-touch input associated with asecond user interface element displayed on the multi-touch surfaceoutside of the region, wherein the second multi-touch input comprisesdragging the second user interface element into the region; and inresponse, generating an additional user interface element disposedwithin the multi-functional touch pad for display within themulti-functional touch pad.
 6. The method of claim 1, furthercomprising: detecting second multi-touch input associated with the firstuser interface element, wherein the second multi-touch input comprisesdragging the first user interface element out of the region; and inresponse, removing the first user interface element from themulti-functional touch pad.
 7. The method of claim 1, furthercomprising: detecting that the application window displayed via thedisplay device is not active; and in response, hiding themulti-functional touch pad from user view.
 8. A computer-readablestorage medium containing a program which, when executed, performs anoperation for configuring a multi-touch surface to display amulti-functional touch pad, the operation comprising: defining a regionon the multi-touch surface for displaying the multi-functional touchpad, wherein the region is proximate to one or more input devicesdetected on the multi-touch surface; generating one or more userinterface elements that are disposed within the multi-functional touchpad, wherein each user interface element is configured to control adifferent operation within an application window displayed via a displaydevice; displaying the multi-functional touch pad in the region on themulti-touch surface; detecting multi-touch input associated with a firstuser interface element disposed within the multi-functional touch pad;and performing a first operation within the application window based onthe multi-touch input.
 9. The computer-readable storage medium of claim8, wherein the first user interface element comprises a tool formanipulating an object within the application window.
 10. Thecomputer-readable storage medium of claim 9, wherein, based on themulti-touch input, the tool is configured to control two or more degreesof freedom of the object.
 11. The computer-readable storage medium ofclaim 8, wherein the first user interface element comprises an icon, andthe first operation is performed when the multi-touch input is receivedat a location on the multi-touch surface corresponding to the locationof the first user interface element.
 12. The computer-readable storagemedium of claim 8, the operation further comprising: detecting secondmulti-touch input associated with a second user interface elementdisplayed on the multi-touch surface outside of the region, wherein thesecond multi-touch input comprises dragging the second user interfaceelement into the region; and in response, generating an additional userinterface element disposed within the multi-functional touch pad fordisplay within the multi-functional touchpad.
 13. The computer-readablestorage medium of claim 8, the operation further comprising: detectingsecond multi-touch input associated with the first user interfaceelement, wherein the second multi-touch input comprises dragging thefirst user interface element out of the region; and in response,removing the first user interface element from the multi-functionaltouch pad.
 14. The computer-readable storage medium of claim 8, theoperation further comprising: detecting that the application windowdisplayed via the display device is not active; and in response, hidingthe multi-functional touch pad from user view.
 15. A system configuredto display a multi-functional touch pad on a multi-touch surface, thesystem comprising: a multi-touch surface; and a processing unit coupledto the multi-touch surface and configured to: define a region on themulti-touch surface for displaying the multi-functional touch pad,wherein the region is proximate to one or more input devices detected onthe multi-touch surface, generate one or more user interface elementsthat are disposed within the multi-functional touch pad, wherein eachuser interface element is configured to control a different operationwithin an application window displayed via a display device, display themulti-functional touch pad in the region on the multi-touch surface,detect multi-touch input associated with a first user interface elementdisposed within the multi-functional touch pad, and perform a firstoperation within the application window based on the multi-touch input.16. The system of claim 15, wherein the first user interface elementcomprises a tool for manipulating an object within the applicationwindow.
 17. The system of claim 16, wherein, based on the multi-touchinput, the tool is configured to control two or more degrees of freedomof the object.
 18. The system of claim 15, wherein the first userinterface element comprises an icon, and the first operation isperformed when the multi-touch input is received at a location on themulti-touch surface corresponding to the location of the first userinterface element.
 19. The system of claim 15, the processing unitfurther configured to: detect second multi-touch input associated with asecond user interface element displayed on the multi-touch surfaceoutside of the region, wherein the second multi-touch input comprisesdragging the second user interface element into the region; and inresponse, generate an additional user interface element disposed withinthe multi-functional touch pad for display within the multi-functionaltouch pad.
 20. The system of claim 15, the processing unit furtherconfigured to: detect second multi-touch input associated with the firstuser interface element, wherein the second multi-touch input comprisesdragging the first user interface element out of the region; and inresponse, remove the first user interface element from themulti-functional touch pad.